{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "2a793b1d-a0a9-404c-ada6-58937227cfce",
   "metadata": {},
   "source": [
    "# Oh dear!\n",
    "\n",
    "If you've got here, then you're still having problems setting up your environment. I'm so sorry! Hang in there and we should have you up and running in no time.\n",
    "\n",
    "Setting up a Data Science environment can be challenging because there's a lot going on under the hood. But we will get there.\n",
    "\n",
    "And please remember - I'm standing by to help out. Message me or email ed@edwarddonner.com and I'll get on the case. The very last cell in this notebook has some diagnostics that will help me figure out what's happening.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98787335-346f-4ee4-9cb7-6181b0e1b964",
   "metadata": {},
   "source": [
    "# Before we begin\n",
    "\n",
    "## Check Cursor extensions\n",
    "\n",
    "Just to confirm that the extensions are installed:\n",
    "- Open extensions (View >> extensions)\n",
    "- Search for python, and when the results show, click on the ms-python one, and Install it if not already installed\n",
    "- Search for jupyter, and when the results show, click on the Microsoft one, and Install it if not already installed  \n",
    "Then View >> Explorer to bring back the File Explorer.\n",
    "\n",
    "## Connect this Kernel:\n",
    "\n",
    "If you see the words `Select Kernel` in a button near the top right of this Window, then press the button!\n",
    "\n",
    "You should see a drop down titled \"Select kernel for..\".\n",
    "\n",
    "Pick the one that begins `.venv python 3.12` - it should be the top choice. You might need to click \"Python Environments\" first.\n",
    "\n",
    "It should now say `.venv (Python 3.12.x)` where it used to say `Select Kernel`.\n",
    "\n",
    "After you click \"Select Kernel\", if there is no option like `.venv (Python 3.12.x)` then please do the following:  \n",
    "1. On Mac: From the Cursor menu, choose Settings >> VS Code Settings (NOTE: be sure to select `VSCode Settings` not `Cursor Settings`);  \n",
    "Or on Windows PC: From the File menu, choose Preferences >> VS Code Settings (NOTE: be sure to select `VSCode Settings` not `Cursor Settings`)  \n",
    "2. In the Settings search bar, type \"venv\"  \n",
    "3. In the field \"Path to folder with a list of Virtual Environments\" put the path to the project root, like C:\\Users\\username\\projects\\agents (on a Windows PC) or /Users/username/projects/agents (on Mac or Linux).  \n",
    "And then try again.\n",
    "\n",
    "## Anaconda interference and Python version horrors\n",
    "\n",
    "Having problems with missing Python versions? Have you ever used Anaconda before? It might be interferring. Quit Cursor, bring up a new command line, and make sure that your Anaconda environment is deactivated:    \n",
    "`conda deactivate`  \n",
    "And if you still have any problems with conda and python versions, it's possible that you will need to run this too:  \n",
    "`conda config --set auto_activate_base false`  \n",
    "And then start a new Terminal / Powershell.  \n",
    "and then from within the agents directory, you should be able to run `uv python list` and see the Python 3.12 version.  \n",
    "And this shouldn't be required, but if you want to be absolutely careful, then now:  \n",
    "1. Delete the folder called '.venv' in the project root directory  \n",
    "2. `uv python uninstall 3.12`\n",
    "3. `uv python install 3.12`\n",
    "4. `uv python list`\n",
    "5. `uv sync`  \n",
    "And surely that is bulletproof!\n",
    "\n",
    "\n",
    "## When the Kernel is connected - checking your internet connection\n",
    "\n",
    "First let's check that there's no VPN or Firewall or Certs problem.\n",
    "\n",
    "Click in the cell below and press Shift+Return to run it.  \n",
    "If this gives you problems, then please try working through these instructions to address:  \n",
    "\n",
    "https://chatgpt.com/share/67fe706a-4bf4-8012-bbfa-29ac870a2c5b\n",
    "\n",
    "If you continue to have certificate issues like this:\n",
    "`ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)`  \n",
    "\n",
    "Then please see the notes near the end of this notebook on how to temporarily bypass SSL for lab work."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d296f9b6-8de4-44db-b5f5-9b653dfd3d81",
   "metadata": {},
   "outputs": [],
   "source": [
    "import urllib.request\n",
    "\n",
    "try:\n",
    "    response = urllib.request.urlopen(\"https://www.google.com\", timeout=10)\n",
    "    if response.status != 200:\n",
    "        print(\"Unable to reach google - there may be issues with your internet / VPN / firewall?\")\n",
    "    else:\n",
    "        print(\"Connected to the internet and can reach Google\")\n",
    "except Exception as e:\n",
    "    print(f\"Failed to connect with this error: {e}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d91da3b2-5a41-4233-9ed6-c53a7661b328",
   "metadata": {},
   "source": [
    "## Another mention of occasional \"gotchas\" for PC people\n",
    "\n",
    "There are 4 snafus on Windows to be aware of:  \n",
    "1. Permissions. Please take a look at this [tutorial](https://chatgpt.com/share/67b0ae58-d1a8-8012-82ca-74762b0408b0) on permissions on Windows\n",
    "2. Anti-virus, Firewall, VPN. These can interfere with installations and network access; try temporarily disabling them as needed\n",
    "3. The evil Windows 260 character limit to filenames - here is a full [explanation and fix](https://chatgpt.com/share/67b0afb9-1b60-8012-a9f7-f968a5a910c7)!\n",
    "4. If you've not worked with Data Science packages on your computer before, you might need to install Microsoft Build Tools. Here are [instructions](https://chatgpt.com/share/67b0b762-327c-8012-b809-b4ec3b9e7be0). A student also mentioned that [these instructions](https://github.com/bycloudai/InstallVSBuildToolsWindows) might be helpful for people on Windows 11.  \n",
    "\n",
    "## And for Mac people\n",
    "\n",
    "1. If you're new to developing on your Mac, you may need to install XCode developer tools. Here are [instructions](https://chatgpt.com/share/67b0b8d7-8eec-8012-9a37-6973b9db11f5).\n",
    "2. As with PC people, Anti-virus, Firewall, VPN can be problematic. These can interfere with installations and network access; try temporarily disabling them as needed"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5190688-205a-46d1-a0dc-9136a42ad0db",
   "metadata": {},
   "source": [
    "# Step 1\n",
    "\n",
    "Try running the next cell (click in the cell under this one and hit shift+return).\n",
    "\n",
    "If this gives an error, then something's up with yoour uv environment or the Kernel of this notebook. Please check Part 3 in the SETUP-PC or SETUP-mac.\n",
    "\n",
    "If **that** doesn't work, then please contact me! I'll respond quickly, and we'll figure it out. Please run the diagnostics (last cell in this notebook) so I can debug. If you used Anaconda, it might be that for some reason your environment is corrupted, in which case the simplest fix is to use the virtualenv approach instead (Part 2B in the setup guides)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7c8c0bb3-0e94-466e-8d1a-4dfbaa014cbe",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Some quick checks that your Conda environment or VirtualEnv is as expected\n",
    "# The Environment Name should be: llms\n",
    "\n",
    "import os\n",
    "venv_name = \"\"\n",
    "\n",
    "virtual_env = os.environ.get('VIRTUAL_ENV')\n",
    "if virtual_env:\n",
    "    print(\"Virtualenv is active:\")\n",
    "    print(f\"Environment Path: {virtual_env}\")\n",
    "    venv_name = os.path.basename(virtual_env)\n",
    "    print(f\"Environment Name: {venv_name}\")\n",
    "\n",
    "if venv_name != \".venv\":\n",
    "    print(\"The uv environment with expected name .venv is not active\")\n",
    "    print(\"Please check Part 3 in the SETUP-PC or SETUP-mac guide\")\n",
    "else:\n",
    "    print(\"So far, so good!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45e2cc99-b7d3-48bd-b27c-910206c4171a",
   "metadata": {},
   "source": [
    "# Step 1.1\n",
    "\n",
    "## It's time to check that the environment is good and dependencies are installed\n",
    "\n",
    "And now, this next cell should run with no output - no import errors.  \n",
    "\n",
    "If any import error, please try SETUP-mac and SETUP-PC Part 3 again, and double check the Kernel button on the top right looks good.."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "6c78b7d9-1eea-412d-8751-3de20c0f6e2f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# This import should work if your environment is active and dependencies are installed!\n",
    "\n",
    "from openai import OpenAI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1e52e937",
   "metadata": {},
   "outputs": [],
   "source": [
    "# More imports!\n",
    "\n",
    "import gradio as gr\n",
    "from dotenv import load_dotenv\n",
    "from agents import Agent, Runner, trace"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b66a8460-7b37-4b4c-a64b-24ae45cf07eb",
   "metadata": {},
   "source": [
    "# Step 2\n",
    "\n",
    "Let's check your .env file exists and has the OpenAI key set properly inside it.  \n",
    "Please run this code and check that it prints a successful message, otherwise follow its instructions.\n",
    "\n",
    "If it isn't successful, then it's not able to find a file called `.env` in the `agents` folder.  \n",
    "The name of the file must be exactly `.env` - it won't work if it's called `my-keys.env` or `.env.doc`.  \n",
    "Is it possible that `.env` is actually called `.env.txt`?\n",
    "\n",
    "Nasty gotchas to watch out for:  \n",
    "- In the .env file, there should be no space between the equals sign and the key. Like: `OPENAI_API_KEY=sk-proj-...`\n",
    "- If you copied and pasted your API key from another application, make sure that it didn't replace hyphens in your key with long dashes  \n",
    "\n",
    "If you're having challenges creating the `.env` file, we can also do it with code! See the cell after the next one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "caa4837e-b970-4f89-aa9a-8aa793c754fd",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "\n",
    "parent_dir = Path(\"..\")\n",
    "env_path = parent_dir / \".env\"\n",
    "\n",
    "if env_path.exists() and env_path.is_file():\n",
    "    print(\".env file found.\")\n",
    "\n",
    "    # Read the contents of the .env file\n",
    "    with env_path.open(\"r\") as env_file:\n",
    "        contents = env_file.readlines()\n",
    "\n",
    "    key_exists = any(line.startswith(\"OPENAI_API_KEY=\") for line in contents)\n",
    "    good_key = any(line.startswith(\"OPENAI_API_KEY=sk-proj-\") for line in contents)\n",
    "    classic_problem = any(\"OPEN_\" in line for line in contents)\n",
    "    \n",
    "    if key_exists and good_key:\n",
    "        print(\"SUCCESS! OPENAI_API_KEY found and it has the right prefix\")\n",
    "    elif key_exists:\n",
    "        print(\"Found an OPENAI_API_KEY although it didn't have the expected prefix sk-proj- \\nPlease double check your key in the file..\")\n",
    "    elif classic_problem:\n",
    "        print(\"Didn't find an OPENAI_API_KEY, but I notice that 'OPEN_' appears - do you have a typo like OPEN_API_KEY instead of OPENAI_API_KEY?\")\n",
    "    else:\n",
    "        print(\"Didn't find an OPENAI_API_KEY in the .env file\")\n",
    "else:\n",
    "    print(\".env file not found in the llm_engineering directory. It needs to have exactly the name: .env\")\n",
    "    \n",
    "    possible_misnamed_files = list(parent_dir.glob(\"*.env*\"))\n",
    "    \n",
    "    if possible_misnamed_files:\n",
    "        print(\"\\nWarning: No '.env' file found, but the following files were found in the llm_engineering directory that contain '.env' in the name. Perhaps this needs to be renamed?\")\n",
    "        for file in possible_misnamed_files:\n",
    "            print(file.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "105f9e0a-9ff4-4344-87c8-e3e41bc50869",
   "metadata": {},
   "source": [
    "## Fallback plan - python code to create the .env file for you\n",
    "\n",
    "Only run the next cell if you're having problems making the .env file.  \n",
    "Replace the text in the first line of code with your key from OpenAI."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ab9ea6ef-49ee-4899-a1c7-75a8bd9ac36b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Only run this code in this cell if you want to have a .env file created for you!\n",
    "\n",
    "# Put your key inside the quote marks\n",
    "make_me_a_file_with_this_key = \"put your key here inside these quotes.. it should start sk-proj-\"\n",
    "\n",
    "# Change this to True if you already have a .env file and you want me to replace it\n",
    "overwrite_if_already_exists = False \n",
    "\n",
    "from pathlib import Path\n",
    "\n",
    "parent_dir = Path(\"..\")\n",
    "env_path = parent_dir / \".env\"\n",
    "\n",
    "if env_path.exists() and not overwrite_if_already_exists:\n",
    "    print(\"There is already a .env file - if you want me to create a new one, change the variable overwrite_if_already_exists to True above\")\n",
    "else:\n",
    "    try:\n",
    "        with env_path.open(mode='w', encoding='utf-8') as env_file:\n",
    "            env_file.write(f\"OPENAI_API_KEY={make_me_a_file_with_this_key}\")\n",
    "        print(f\"Successfully created the .env file at {env_path}\")\n",
    "        if not make_me_a_file_with_this_key.startswith(\"sk-proj-\"):\n",
    "            print(f\"The key that you provided started with '{make_me_a_file_with_this_key[:8]}' which is different to sk-proj- is that what you intended?\")\n",
    "        print(\"Now rerun the previous cell to confirm that the file is created and the key is correct.\")\n",
    "    except Exception as e:\n",
    "        print(f\"An error occurred while creating the .env file: {e}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0ba9420d-3bf0-4e08-abac-f2fbf0e9c7f1",
   "metadata": {},
   "source": [
    "# Step 3\n",
    "\n",
    "Now let's check that your API key is correct set up in your `.env` file, and available using the dotenv package.\n",
    "Try running the next cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0ee8e613-5a6e-4d1f-96ef-91132da545c8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# This should print your API key to the output - please follow the instructions that get printed\n",
    "\n",
    "import os\n",
    "from dotenv import load_dotenv\n",
    "load_dotenv(override=True)\n",
    "\n",
    "api_key = os.getenv(\"OPENAI_API_KEY\")\n",
    "\n",
    "if not api_key:\n",
    "    print(\"No API key was found - please try Kernel menu >> Restart Kernel And Clear Outputs of All Cells\")\n",
    "elif not api_key.startswith(\"sk-proj-\"):\n",
    "    print(f\"An API key was found, but it starts with {api_key[:8]} rather than sk-proj- please double check this is as expected.\")\n",
    "elif api_key.strip() != api_key:\n",
    "    print(\"An API key was found, but it looks like it might have space or tab characters at the start or end - please remove them\")\n",
    "else:\n",
    "    print(\"API key found and looks good so far!\")\n",
    "\n",
    "if api_key:\n",
    "    problematic_unicode_chars = ['\\u2013', '\\u2014', '\\u201c', '\\u201d', '\\u2026', '\\u2018', '\\u2019']\n",
    "    forbidden_chars = [\"'\", \" \", \"\\n\", \"\\r\", '\"']\n",
    "    \n",
    "    if not all(32 <= ord(char) <= 126 for char in api_key):\n",
    "        print(\"Potential problem: there might be unprintable characters accidentally included in the key?\")\n",
    "    elif any(char in api_key for char in problematic_unicode_chars):\n",
    "        print(\"Potential problem: there might be special characters, like long hyphens or curly quotes in the key - did you copy it via a word processor?\")\n",
    "    elif any(char in api_key for char in forbidden_chars):\n",
    "        print(\"Potential problem: there are quote marks, spaces or empty lines in your key?\")\n",
    "    else:\n",
    "        print(\"The API key contains valid characters\")\n",
    "    \n",
    "print(f\"\\nHere is the key --> {api_key} <--\")\n",
    "print()\n",
    "print(\"If this key looks good, please press Clear All Outputs so that your key is no longer displayed here!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f403e515-0e7d-4be4-bb79-5a102dbd6c94",
   "metadata": {},
   "source": [
    "## It should print some checks including something like:\n",
    "\n",
    "`Here is the key --> sk-proj-blahblahblah <--`\n",
    "\n",
    "If it didn't print a key, then hopefully it's given you enough information to figure this out. Or contact me!\n",
    "\n",
    "There is a final fallback approach if you wish: you can avoid using .env files altogether, and simply always provide your API key manually.  \n",
    "Whenever you see this in the code:  \n",
    "`openai = OpenAI()`  \n",
    "You can replace it with:  \n",
    "`openai = OpenAI(api_key=\"sk-proj-xxx\")`  \n",
    "where obviously you replace sk-proj-xxx with your actual key."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42afad1f-b0bf-4882-b469-7709060fee3a",
   "metadata": {},
   "source": [
    "# Step 4\n",
    "\n",
    "Now run the below code and you will hopefully see that GPT can handle basic arithmetic!!\n",
    "\n",
    "If not, see the cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cccb58e7-6626-4033-9dc1-e7e3ff742f6b",
   "metadata": {},
   "outputs": [],
   "source": [
    "from openai import OpenAI\n",
    "from dotenv import load_dotenv\n",
    "load_dotenv(override=True)\n",
    "\n",
    "my_api_key = os.getenv(\"OPENAI_API_KEY\")\n",
    "\n",
    "print(f\"Using API key --> {my_api_key} <--\")\n",
    "\n",
    "openai = OpenAI()\n",
    "completion = openai.chat.completions.create(\n",
    "    model='gpt-4o-mini',\n",
    "    messages=[{\"role\":\"user\", \"content\": \"What's 2+2?\"}],\n",
    ")\n",
    "print(completion.choices[0].message.content)\n",
    "print(\"Now go to Edit menu >> Clear Cell Output to remove the display of your key.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81046a77-c359-4388-929f-ffc8ad5cb93c",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "## If the key was set correctly, and this still didn't work\n",
    "\n",
    "### If there's an error from OpenAI about your key, or a Rate Limit Error, then there's something up with your API key!\n",
    "\n",
    "First check [this webpage](https://platform.openai.com/settings/organization/billing/overview) to make sure you have a positive credit balance.\n",
    "OpenAI requires that you have a positive credit balance and it has minimums, typically around $5 in local currency. My salespitch for OpenAI is that this is well worth it for your education: for less than the price of a music album, you will build so much valuable commercial experience. But it's not required for this course at all; the guides have instructions to call free open-source models via Ollama whenever we use OpenAI.\n",
    "\n",
    "OpenAI billing page with credit balance is here:   \n",
    "https://platform.openai.com/settings/organization/billing/overview  \n",
    "OpenAI can take a few minutes to enable your key after you top up your balance.  \n",
    "A student outside the US mentioned that he needed to allow international payments on his credit card for this to work.  \n",
    "\n",
    "It's unlikely, but if there's something wrong with your key, you could also try creating a new key (button on the top right) here:  \n",
    "https://platform.openai.com/api-keys\n",
    "\n",
    "### Check that you can use gpt-4o-mini from the OpenAI playground\n",
    "\n",
    "To confirm that billing is set up and your key is good, you could try using gtp-4o-mini directly:  \n",
    "https://platform.openai.com/playground/chat?models=gpt-4o-mini\n",
    "\n",
    "### If there's a cert related error\n",
    "\n",
    "If you encountered a certificates error like:  \n",
    "`ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)`  \n",
    "Then please replace:\n",
    "`openai = OpenAI()`  \n",
    "with:  \n",
    "`import httpx`  \n",
    "`openai = OpenAI(http_client=httpx.Client(verify=False))`  \n",
    "And also please replace:  \n",
    "`requests.get(url, headers=headers)`  \n",
    "with:  \n",
    "`requests.get(url, headers=headers, verify=False)`  \n",
    "And if that works, you're in good shape. You'll just have to change the labs in the same way any time you hit this cert error.  \n",
    "This approach isn't OK for production code, but it's fine for our experiments. You may need to contact IT support to understand whether there are restrictions in your environment.\n",
    "\n",
    "## If all else fails:\n",
    "\n",
    "(1) Try pasting your error into ChatGPT or Claude! It's amazing how often they can figure things out\n",
    "\n",
    "(2) Try creating another key and replacing it in the .env file and rerunning!\n",
    "\n",
    "(3) Contact me! Please run the diagnostics in the cell below, then email me your problems to ed@edwarddonner.com\n",
    "\n",
    "Thanks so much, and I'm sorry this is giving you bother!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc83f944-6ce0-4b5c-817f-952676e284ec",
   "metadata": {},
   "source": [
    "# Gathering Essential Diagnostic information\n",
    "\n",
    "## Please run this next cell to gather some important data\n",
    "\n",
    "Please run the next cell; it should take a minute or so to run. Most of the time is checking your network bandwidth.\n",
    "Then email me the output of the last cell to ed@edwarddonner.com.  \n",
    "Alternatively: this will create a file called report.txt - just attach the file to your email."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "248204f0-7bad-482a-b715-fb06a3553916",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run my diagnostics report to collect key information for debugging\n",
    "# Please email me the results. Either copy & paste the output, or attach the file report.txt\n",
    "\n",
    "from diagnostics import Diagnostics\n",
    "Diagnostics().run()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
